GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8) - #2122

matrix.config.conc for multi-node agentic entries is a JSON array
(chunked concurrencies per allocation), but sweep-multi-node-agentic
passed it directly into benchmark-multinode-tmpl.yml's `conc` input,
which is declared `type: string` ("First concurrency for
agentic-coding scenarios; CONC_LIST carries the full batch"). GitHub
Actions' reusable-workflow input validator rejects a sequence value
for a string-typed input at evaluation time, so the whole job failed
to even load:

  evaluate reusable workflow inputs: .github/workflows/run-sweep.yml
  (Line: 554, Col: 19): A sequence was not expected

Since the job never materializes when this happens, sweep-multi-node-
agentic silently disappeared from run summaries entirely instead of
showing as failed. Slice to the first element (matching the intended
"first concurrency" semantics and the same fix already applied
elsewhere, e.g. PR #2122) to restore a scalar value.

Co-authored-by: Cursor <cursoragent@cursor.com>

matrix.config.conc for multi-node agentic entries is a JSON array
(chunked concurrencies per allocation), but sweep-multi-node-agentic
passed it directly into benchmark-multinode-tmpl.yml's `conc` input,
which is declared `type: string` ("First concurrency for
agentic-coding scenarios; CONC_LIST carries the full batch"). GitHub
Actions' reusable-workflow input validator rejects a sequence value
for a string-typed input at evaluation time, so the whole job failed
to even load:

  evaluate reusable workflow inputs: .github/workflows/run-sweep.yml
  (Line: 554, Col: 19): A sequence was not expected

Since the job never materializes when this happens, sweep-multi-node-
agentic silently disappeared from run summaries entirely instead of
showing as failed. Slice to the first element (matching the intended
"first concurrency" semantics and the same fix already applied
elsewhere, e.g. PR #2122) to restore a scalar value.

Co-authored-by: Cursor <cursoragent@cursor.com>

…2170)

* [AMD][AgentX] Add DeepSeek V4 MI355X agentic disagg

* run-sweep: fix broken conc-list interpolation for multi-node agentic

matrix.config.conc for multi-node agentic is already a JSON array
(e.g. [16,32,64] from generate_sweep_configs.py). Wrapping it in a
string literal as '[${{ matrix.config.conc }}]' can't correctly
interpolate an array into a string, producing a malformed conc-list
so fromJson(inputs.conc-list) in the reusable template ended up with
an empty/wrong CONC_LIST. Use toJson(matrix.config.conc) instead,
matching the pattern already used for the other two conc-list inputs
in this file.

Co-authored-by: Cursor <cursoragent@cursor.com>

* bump image to rocm/sgl-dev:sglang-0.5.14-rocm720-mi35x-mori-0706

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* run-sweep: fix sequence-typed conc input for multi-node agentic

matrix.config.conc for multi-node agentic entries is a JSON array
(chunked concurrencies per allocation), but sweep-multi-node-agentic
passed it directly into benchmark-multinode-tmpl.yml's `conc` input,
which is declared `type: string` ("First concurrency for
agentic-coding scenarios; CONC_LIST carries the full batch"). GitHub
Actions' reusable-workflow input validator rejects a sequence value
for a string-typed input at evaluation time, so the whole job failed
to even load:

  evaluate reusable workflow inputs: .github/workflows/run-sweep.yml
  (Line: 554, Col: 19): A sequence was not expected

Since the job never materializes when this happens, sweep-multi-node-
agentic silently disappeared from run summaries entirely instead of
showing as failed. Slice to the first element (matching the intended
"first concurrency" semantics and the same fix already applied
elsewhere, e.g. PR #2122) to restore a scalar value.

Co-authored-by: Cursor <cursoragent@cursor.com>

* agentic: run one task per concurrency for multi-node sweeps

Multi-node agentic sweeps batched up to 4 concurrencies per SLURM
allocation, running them sequentially against one shared server session.
A slow/hung conc could block the rest of the batch from ever producing
results, which is why run #6719 only reported c16 despite a
16/32/48/64 conc-list. Drop the batch size to 1 so each concurrency
gets its own task/allocation, matching the granularity already used
for single-node agentic sweeps.

Cherry-picked from backup/agentx-v1.0-rebase-pre-upstream-rewrite-20260710
(082a59d), adapted for the current test suite.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Bump image to latest upstream image with 4 PR fixes included.

* update con-list

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* setup_deps: remove disabled decode_tp_queue_agree patch

patch_decode_tp_queue_agree()'s invocation was already commented out
(dead code with no runtime effect), and the reference-only
patches/decode_tp_queue_agree.patch it mirrored had no other callers.
Drop both plus the README bullet pointing at it.

Co-authored-by: Cursor <cursoragent@cursor.com>

* setup_deps: broaden GLM transformers gate; disable prefill bootstrap-desync patch

install_transformers_glm5() was gated on an exact MODEL_NAME ==
"GLM-5-FP8" match; broaden to any model name containing "GLM" so other
GLM variants pick up the same glm_moe_dsa transformers fix.

Also disable patch_disagg_prefill_bootstrap_desync's invocation
(commented out, matching the already-disabled decode_tp_queue_agree
pattern removed earlier).

Co-authored-by: Cursor <cursoragent@cursor.com>

* agentic: node-0 sibling benchmark-client container for DSv4 sweeps (#2147)

* agentic: add node-0 sibling benchmark-client container for DSv4 sweeps

Port the "same-node sibling container" client mode from ROCm/InferenceY: when
CLIENT_IMAGE is set (and no CLIENT_NODES), node 0 launches the aiperf trace
replay in its own pre-baked container via the host docker socket, instead of
rebuilding the aiperf venv inside the server container and running it
co-located. This keeps the client's CPU-heavy tokenize/aggregate work off the
sglang scheduler + router, which inflates TTFT/E2E and lowers throughput under
agentic concurrency.

- server_sglang.sh: add IS_AGENTIC_RUN and a CLIENT_IMAGE sibling-container
  branch that writes client.env and docker-runs the client against the local
  router (--network host).
- job.slurm: define CLIENT_CONT_NAME; when CLIENT_IMAGE is set, mount the host
  docker socket + CLI into the server container, forward
  HOST_REPO_DIR/HOST_MODEL_DIR/HOST_BENCH_LOGS/CLIENT_CONT_NAME, pre-pull the
  client image, and clean up the client container on teardown.
- amd-master.yaml: enable the sibling client on
  dsv4-fp4-mi355x-sglang-disagg-agentic-hicache via CLIENT_IMAGE.

The separate-client-NODE mode is intentionally not ported.

* agentic: use server image for sibling client so upstream CI can pull it

The pre-baked rocm/pytorch-private aiperf client image is not pullable by
upstream CI runners. Reuse the (public) server image as CLIENT_IMAGE and build
aiperf on the fly from /workspace/utils/aiperf, matching the co-located path.
Gate the pre-baked-venv env (AIPERF_USE_PREBUILT / AIPERF_VENV) behind an
optional CLIENT_AIPERF_VENV so a real pre-baked client image can still opt in.

* enable log info and extend timeout

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* fix: address PR 2170 review comments

Update the DSv4 MI355X agentic perf-changelog entry to point at PR 2170 with an accurate description, and move hf_dir into the DeepSeek-V4-Pro YAML block so the SGLang model path extractor can read it.

中文：修复 PR 2170 的 review 反馈：将 DSv4 MI355X agentic 的 perf-changelog 条目改为指向 PR 2170 并更新为准确描述，同时把 hf_dir 移入 DeepSeek-V4-Pro YAML 配置块，确保 SGLang 模型路径提取逻辑可以读取。

* setup_deps: remove prefill bootstrap-desync patch entirely

The disaggregation-prefill bootstrap-desync patch was already disabled
(commented out); drop the function, its comment block, and the stale
invocation now that the fix is carried upstream in the pinned image.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix perf changelog

* consolidate sglang agentic envs

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* select models.yaml recipe by IS_AGENTIC (AgentX/DI variants)

Resolve the models.yaml entry per run type: agentic runs (IS_AGENTIC) use
the '<model>-AgentX' recipe, non-agentic disaggregated runs use '<model>-DI',
falling back to the bare model name when no variant key exists. Apply the same
logic in both server_sglang.sh (lookup) and job.slurm (validation guard) so a
bare MODEL_NAME (e.g. DeepSeek-V4-Pro) correctly maps to the suffixed key.
Add DeepSeek-V4-Pro-DI and rename the agentic recipe to DeepSeek-V4-Pro-AgentX.

Co-authored-by: Cursor <cursoragent@cursor.com>

* DPTP enabling.

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* tune hicache ratio, router barrier timeout, and agentic conc sweep

- server_sglang.sh: lower default HICACHE_RATIO 24->5 (cut host-pool
  pin/registration startup cost) and raise router health barrier
  timeout 1800->3000 so a slow server bring-up no longer trips the
  cross-node router wait first.
- amd-master.yaml: restore conc 32 on the non-DP-attn (TP8) point and
  comment out the DP-attn c64/128 sweep block.
- models.yaml: drop explicit max_total_tokens override for the AgentX
  dp variant (let it derive from context_length).

Co-authored-by: Cursor <cursoragent@cursor.com>

* raise hicache ratio to 16

Signed-off-by: Theresa Shan <theresa.shan@amd.com>

* fix: restore qwen3.5 changelog entry removed during rebase

perf-changelog.yaml must only gain entries; re-adding the upstream
qwen3.5-fp8-gb300-dynamo-sglang block fixes process_changelog.py.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: validate agentic hicache master config for changelog gate

Use kv-offload-backend object form and add kv-p2p-transfer: mori for
the disaggregated dsv4-fp4-mi355x-sglang-disagg-agentic-hicache entry.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: wire KV_P2P_TRANSFER env into SGLang launch flags

Override models.yaml --disaggregation-transfer-backend when
kv-p2p-transfer from amd-master.yaml is set via CI, and forward
KV_P2P_TRANSFER to the agentic client env for result metadata.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: drop max_total_tokens from DSv4 AgentX prefill no_dp recipe

Let SGLang auto-size the prefill KV pool instead of pinning 1M tokens.

Co-authored-by: Cursor <cursoragent@cursor.com>

* tune: raise router barrier timeout to 3000s for worker nodes

Give prefill/decode workers more time to wait for the node-0 proxy
during long agentic server startup.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: fix sweep config fixtures for kv-p2p-transfer validation

Add kv-p2p-transfer to disagg multinode test configs and use
KVOffloadBackendMetadata dict form for kv-offload-backend fields.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor: centralize sync barrier timeouts for DSV4 startup

Extract SYNC_BARRIER_TIMEOUT (3000s for DeepSeek-V4, 1800s otherwise) and
apply it to server-up, router readiness, and prefill/decode proxy barriers.
Also lower the default HICACHE_RATIO from 16 to 5.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: forward KV offload backend metadata to agentic client

Pass KV_OFFLOAD_BACKEND_METADATA into server containers via job.slurm and
forward it to sibling client containers in server_sglang client.env so
process_agentic_result can write aggregate JSON. Derive KV_OFFLOAD_BACKEND
from metadata in the DSv4 agentic recipe when unset.

Co-authored-by: Cursor <cursoragent@cursor.com>

* revert: drop metadata-derived KV_OFFLOAD_BACKEND from DSv4 recipe

CI already sets KV_OFFLOAD_BACKEND from the workflow; keep the job.slurm and
server_sglang metadata forwarding fixes without the redundant recipe logic.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: quote KV_OFFLOAD_BACKEND_METADATA in docker run env forwarding

toJson() pretty-prints with embedded newlines, so the unquoted expansion in
job.slurm's DOCKER_ENV_COMMON was word-split across the docker run command,
corrupting its argument list ("docker: invalid reference format", exit 125).
Quote the value so the whole JSON stays one token.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix client.env generation to flatten multi-line env values

docker run --env-file requires strict single-line KEY=VALUE entries.
KV_OFFLOAD_BACKEND_METADATA carries pretty-printed multi-line JSON,
which split into invalid env-file lines (e.g. '"name": "hicache",')
and aborted the sibling client container launch with "docker: invalid
env file ... contains whitespaces". Strip embedded newlines before
writing each value so the JSON stays valid but single-line.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Re-serialize KV_OFFLOAD_BACKEND_METADATA as compact JSON instead of stripping newlines

Naive newline-stripping could silently corrupt a value if a JSON string
ever contained a literal embedded newline (merging tokens across the
line break). Round-tripping through json.loads/json.dumps guarantees a
correct compact single-line representation for client.env, and fails
loudly with a clear error if the value isn't valid JSON.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix regression: skip JSON round-trip for empty KV_OFFLOAD_BACKEND_METADATA

job.slurm always forwards KV_OFFLOAD_BACKEND_METADATA into the server
container, even as "" when KV offloading is disabled (the common
default case). The prior commit unconditionally ran it through
json.loads/json.dumps, which raised JSONDecodeError on that empty
string and hard-failed every agentic client-container launch without
KV offloading. Only attempt the JSON round-trip when the value is
non-empty and not "null", matching how
optional_kv_offload_backend_metadata() already treats those as
"no metadata" rather than invalid JSON.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(changelog): append PR entry after main

* fix(changelog): omit null optional metadata

* fix(agentx): set DeepSeek reasoning effort high

* Remove stale patches/README.md documenting an already-retired mori_conn.py overlay

mori_conn.py and the job.slurm auto-apply block it documented were
already deleted upstream (retire MoRI conn.py PD-disagg overlay,
aca9cee) once the real sglang fix landed in published images. A
later unrelated commit (183a987) accidentally recreated this README
from scratch while only intending to drop a bullet about a different,
already-removed patch (decode_tp_queue_agree.patch), leaving stale
docs pointing at a file and job.slurm block that no longer exist.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(agentx): enforce agentic DRAM offload budget for multinode disagg

Multinode agentic entries hardcoded total-cpu-dram-gb=0, so the DRAM
KV-offload pool was never bounded at runtime (HiCache fell back to a
relative --hicache-ratio). Compute a real per-node budget from the
prefill worker (the only KV offloader today) and wire it through to a
--hicache-size cap.

- generate_sweep_configs: fold multinode into agentic_dram_offload_gb;
  budget the prefill worker's per-node GPU footprint via worker_gpus_per_node,
  which rejects node-misaligned topologies instead of silently truncating.
- validation: add total-cpu-dram-gb to MultiNodeAgenticMatrixEntry.
- run-sweep: forward total-cpu-dram-gb to the multi-node agentic dispatch.
- server_sglang.sh: size prefill HiCache via absolute --hicache-size
  (budget / ranks-per-node / host-pools), falling back to --hicache-ratio
  when no budget is provided; drop the dead HICACHE_TOTAL_CPU_DRAM_GB default.

* Fix indentation of dsv4-fp4-b200-vllm-agentic entry in perf-changelog.yaml

The config-keys entry was missing its leading indentation, breaking YAML list alignment with sibling entries.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(agentx): skip --hicache-size for DeepSeek V4 in multinode HiCache

DeepSeek V4's hybrid HiCache pool assembler rejects --hicache-size and
requires --hicache-ratio, so the absolute per-node DRAM budget cannot be
applied to it. Restrict the --hicache-size enforcement to non-V4 models;
V4 falls back to --hicache-ratio. Budget stays recorded in the config.

中文：多节点 HiCache 对 DeepSeek V4 跳过 --hicache-size。DeepSeek V4 的
混合 HiCache 池不支持 --hicache-size、必须用 --hicache-ratio,因此绝对
的每节点 DRAM 预算无法作用于它。将 --hicache-size 限制为非 V4 模型;
V4 回退到 --hicache-ratio。预算仍记录在 config 中。

---------

Signed-off-by: Theresa Shan <theresa.shan@amd.com>
Co-authored-by: Cam Quilici <cjquilici@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: AMD-yanfeiwang <yanfei.wang@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: billishyahao <bill.he@amd.com>